The latent ordinary differential equation architecture consists of a variational auto-encoder structure with the addition of a neural ordinary differential equation in its latent space, thus forming a trainable latent dynamical system. To build a comprehensive data-driven model of the time series’ originating process, we aim to capture its unknown dynamic properties with this latent system. The auto-encoder structure here allows for flexibility by enabling access to an arbitrary latent dimension and letting the model establish latent variables in an unsupervised fashion, instead of constraining it to variables available in the time series. However, this flexibility can prove to be detrimental to the overall model. By default, the criterion used to train the latent ODE does not explicitly consider the neural ODE’s output. This can result in the decoder overstepping its intended purpose and becoming the dominant contributor to the reconstruction, while downplaying the role of the neural ODE. In this case, the model can reconstruct the time series without modeling the potential underlying dynamic and will struggle to fulfill auxiliary objectives like temporal extrapolation. In experiments made using data generated with an oscillator, we went from a model managing periodicity to one that does not by adding a single layer to the decoder, with no apparent differences in validation losses. In real-world scenarios, distinguishing dynamically accurate from inaccurate models is a crucial task. We want to ensure that the neural ODE stays the primary actor of the reconstruction, as it is the sole component fit to model an actual dynamic. While the most straightforward method would be to restrain the decoder’s potency by limiting its architecture, this would also hinder the overall model’s capacity to form an adequate latent space. We are currently investigating a solution relying on matching topological properties between latent and data spaces. First results from such methods using generated data are promising, and further try-outs on real-world applications are planned in the upcoming months.